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Abstract 


This  researxh  focused  on  which  mode  of  presentation,  tabular  or  graphical 
would  be  more  effective  in  aiding  a  program  manager  in  his  evaluation  of  a 
contractor's  performance  when  cost/schedule  control  systems  criteria  is 
used.  Effectiveness  was  measured  in  terms  of  decision  quality,  speed, 
confidence,  and  difficulty.  A  review  of  the  literature  in  this  area  identified 
several  important  variables  that  might  effect  an  individual's  performance 
when  a  particular  mode  of  presentation  is  used.  These  variables  include  the 
task  to  be  accomplished,  field  independence/dependence,  and  experience. 

Two  test  scenarios  were  designed  to  test  the  individual  s  ability  to  detect 
trends,  come  up  with  precise  numbers,  and  evaluate  a  contractor  s 
performance  using  graphical  or  tabular  information.  The  scenarios  were 
administered  to  individuals  who  are  or  will  be  involved  in  the  acquisition 

r  JO 

process.  ’Along  with  the  tost  socnaf ios,  the  Group  Embedded  Figures  Test 

ike;*' 

was  administered  to  the  subjects  to  determine  if  the  individual  was  field 
independent  or  dependent.  A  questionnaire  was  also  developed  to  collect 
background  information  on  the  sample  population. 

A  statistical  analysis  of  the  data  collected  indicated  that  the  task  to  be 
accomplished  had  a  large  effect  on  which  mode  of  presentation  was  most 
effective.  Tabular  information  was  best  when  accuracy  was  desired,  and 
graphical  data  was  best  when  trends  needed  to  be  analyzed. 


AN  INVESTIGATION  INTO  THE  EFFECT  OF  THE  MODE  OF  PRESENTATION  ON 
CONTRACT  EVALUATION  WHEN  COST/SCHEDULE  CONTROL  SYSTEM  CRITERIA 

IS  USED 


I.  Introduction 


Background 

Information  is  a  powerful  and  important  business  asset  which  influences 
the  health  and  wealth  of  a  corporation  (5:29).  Since  the  invention  of  the 
computer,  businesses  have  been  trying  to  find  new  and  better  ways  to  use 
computers  to  aid  in  the  processing  of  information.  Computer  based 
information  systems  are  designed  to  collect,  store,  retrieve,  process,  and 
present  information  (12:309).  Information  systems  can  be  defined  as 
"planned  and  organized  approaches  to  supplying  executives  with  intelligence 
aids  that  facilitate  the  managerial  process"  (10:21).  Information  systems 
sort  and  assemble  raw  data  in  order  to  create  the  desired  information.  The 
key  is  that  the  information  must  be  in  a  form  that  is  useful  to  management. 
In  short,  the  manager  needs  to  be  able  to  find  the  information  necessary  to 
make  decisions,  spot  problem  areas,  and  perform  other  managerial  tasks. 
Therefore,  one  of  the  major  decisions  facing  an  information  system  designer 
is  what  mode  of  presentation  is  the  best  one  for  displaying  the  information. 
Mode  of  presentation  refers  to  the  way  information  is  presented  to  the  user. 
The  mode  of  presentation  needs  to  correspond  to  the  type  of  decision  being 
made,  and  the  needs  of  the  user.  The  data  must  be  presented  in  an 
understandable  format  that  allows  managers  to  easily  find  the  information 
they  need  to  accomplish  their  task  without  having  to  "hunt"  for  information. 


In  the  past,  the  most  popular  form  of  information  presentation  has  been 
tabular  data.  However,  recently,  information  system  designers  have  become 
more  interested  in  graphics  as  a  mode  of  presentation  Decreased  hardware 
costs,  the  availability  of  easy-to-use  software,  and  the  availability  of 
graphics  technology  for  office  workstations  have  all  contributed  to  this 
increased  interest  (6:43)  Improvements  in  hardware  have  made  it  possible 
to  view  graphs  in  color  as  well  as  black  and  white  Software  enhancements 
allow  managers  to  highlight  parts  of  the  graph  for  emphasis  (13: 10). 
Graphics,  are  not  new;  however,  the  computer  technology  now  exists  that 
enables  them  to  be  produced  faster,  cheaper,  and  by  the  end  user  (9:16) 
Proponents  of  computer  graphics  claim  that  they  will  increase  managerial 
productivity  in  the  areas  of  information  recall,  decision  speed  and  quality 
(9.15).  However,  the  research  in  this  area  does  not  fully  support  these 
claims.  The  research  that  has  been  done  has  produced  mixed  results.  Some 
studies  favored  graphics,  while  others  favored  tables.  More  research  needs 
to  be  done  in  order  to  achieve  some  conclusive  results.  The  fundamental 
issue  that  needs  to  be  researched  is  the  ability  of  managers  to  accurately 
interpret  the  information  presented  graphically  and  make  quality  decisions 
based  on  that  information  (6:464). 

When  choosing  a  mode  of  presentation,  an  information  systems  designer 
should  ask  the  following  questions: 

1.  What  are  the  needs  of  the  user? 

2.  What  is  the  user's  background? 

3.  What  is  the  task  to  be  accomplished? 

4.  Which  mode  of  presentation  will  facilitate  the  user's 
understanding? 
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The  data  should  be  presented  in  such  a  way  that  managers  can  understand 
it,  find  the  information  needed  to  make  decisions,  spot  problem  areas,  and 
formulate  and  evaluate  alternative  courses  of  action.  In  general,  information 
should  be  displayed  in  a  way  that  facilitates  managerial  activities. 

Research  Problem 

In  order  to  choose  the  correct  mode  of  presentation  to  use,  the 
‘  information  system  designer  needs  to  determine  the  best  way  to  display  the 
information.  Research  needs  to  be  done  to  determine  when  to  use  tabular 
information  and  when  to  use  graphical  information.  Graphics  may  help 
managers  make  better  decisions  in  some  situations,  while  tabular 
information  will  be  better  in  others.  The  past  research  suggests  that  the 
situation  is  a  determining  factor  when  choosing  what  mode  of  presentation 
to  use  (3:67,6:475).  For  instance,  graphics  appear  to  better  at  detecting 
trends  and  making  comparisons.  Whereas,  tabular  data  is  best  when  the 
user  needs  precise  numbers,  because  tabular  data  has  been  shown  to  lead  to 
better  accuracy  (3:67).  This  study  examined  which  mode  of  presentation, 
tabular  or  graphical,  is  best  suited  to  aid  program  managers  in  evaluating  a 
contractor's  performance.  This  situation  requires  a  program  manager  to 
detect  trends  and  compare  numbers  to  accurately  assess  the  contractor  s 
performance.  This  situation  was  picked  because  several  program  managers, 
working  at  the  Aeronautical  System  s  Division  at  Wright  Patterson  A.F.B.  did 
not  like  the  way  the  information  was  being  presented  to  them  from  the 
contractor.  The  cost  data  was  coming  in  lists,  and  program  managers  were 
having  a  difficult  time  determining  whether  the  contractor  was  over  or 
under  cost,  and  ahead  or  behind  schedule.  Terms  are  defined  below. 


Research  Question.  When  using  cost/schedule  control  system  criteria 
(C/SCSC)  to  evaluate  a  contractor  s  performance,  which  mode  of  presentation, 
tabular  or  graphical,  will  be  more  effective  in  aiding  program  managers  in 
terms  of  decision  quality,  speed,  confidence,  and  difficulty?  Will  graphical  or 
tabular  information  lead  to  a  better  understanding  of  the  situation? 

Scope  and  Limitations 

This  study  compared  the  effectiveness  of  tabular  and  graphical 
information  presentation  in  aiding  a  program  manager  in  his  evaluation  of  a 
contractor  s  performance.  The  test  subjects  were  given  information 
pertaining  to  a  contract.  The  information  was  displayed  in  a  type  written 
report  either  graphically  or  in  tabular  form.  They  were  then  asked  to 
evaluate  the  performance  of  the  contractor  based  on  the  information 
supplied.  By  choosing  this  situation,  the  population  will  be  limited  to  those 
individuals  who  have  some  knowledge  of  C/SCSC. 

There  are  several  dependent  and  independent  variables  which  can  be 
studied  concerning  information  presentation  methods.  Based  on  past 
research  the  following  appear  to  be  the  most  important.  For  this  reason  they 
were  examined  by  this  study: 

Dependent  Variables  -  decision  quality  and  speed,  user  confidence,  and 
decision  difficulty; 

Independent  Variables  -  field  dependence/independence,  user 
background,  and  mode  of  presentation. 

Definition  of  Terms 

Following  are  the  definitions  of  the  key  terms  used  in  this  report: 

1.  Field  dependence/independence:  "a  cognitive  style  which  reflects 


whether  an  individual  is  bound  by  external  referents  or  can  make  use  of 
internal  referents  in  structuring  cognitions,"  (17:967). 


2.  Decision  quality:  decision  quality  for  this  report  was  measured  by  how 
accurately  an  individual  interpreted  the  contractor's  performance. 

3.  Decision  speed:  how  much  time  it  takes  an  individual  to  evaluate  the 
contractor  s  performance. 

4.  Demographics:  background  information  about  an  individual,  such  as  age. 
sex,  job  experience,  etc. 

3.  Cost/Schedule  Control  System  Criteria:  a  set  of  standards  that  a 
contractors  management  system  must  meet  when  developing  a  major 
defense  system  for  the  Department  of  Defense  (1:262). 

6.  Mode  of  presentation:  the  way  information  is  portrayed  in  a  report.  In 
this  report  it  rerers  to  either  graphs  or  tables. 

7  Effectiveness:  For  this  report  the  effectiveness  will  measured  in  terms 
of  how  well  the  mode  of  presentation  aided  the  individual  in  evaluating 
the  contractor  s  performance  in  terms  of  decision  quality  and  speed. 


5 


1 1 .  Literature  Review 


The  goal  of  information  systems  designers  is  to  produce  a  system  that 
will  aid  managers  in  making  decisions,  spotting  problems,  evaluating 
alternative  courses  of  action,  and  accomplishing  other  managerial  tasks 
(4:1348).  Recently,  designers  have  been  paying  attention  to  the  mode  of 
data  presentation.  Is  one  form  of  information  presentation  easier  to 
understand?  Can  information  be  presented  in  such  a  way  that  it  will  help 
managers  make  better  decisions?  Spot  problem  areas  quickly?  Declining 
hardware  costs,  easy-to-use  software,  and  the  availability  of  graphic 
technology  for  office  work  stations  have  increased  the  interest  of  designers 
in  computer  generated  graphics  as  a  mode  of  information  presentation 
(6:463).  Proponents  of  computer  graphics  claim  the  following: 

1.  Graphics  lead  to  more  effective  analysis  and  decision  making. 

2.  Graphics  help  managers  spot  problems. 

3.  Graphics  make  tasks  easier. 

4.  Graphics  are  more  convincing  in  presentations. 

5.  A  picture  is  worth  a  thousand  words  ( 1 5: 1 ). 

They  feel  graphical  representation  of  data,  originally  presented  in  tables, 
may  allow  managers  to  compare  data  values,  observe  patterns,  and  detect 
trends  more  easily  (6.464).  However,  research  does  not  support  their  claims. 
Graphs  have  been  found  superior  to  tables  is  some  situations  but  not  in 
others.  More  research  is  needed  before  these  claims  can  be  considered  fact 
or  fiction  (15:3-4).  Past  studies  comparing  graphical  and  tabular  information 


have  concentrated  in  two  areas:  graphs  versus  tables  and  individual 
differences  which  influence  a  manager  s  ability  to  use  computer  graphics. 

Graphical  vs  Tabular  Information 

The  features  most  studied  by  researchers  have  been  interpretation 
accuracy,  problem  comprehension,  and  task  performance  (6:468).  Because 
the  traditional  method  of  display  is  tables,  these  studies  have  compared 
graphical  to  tabular  information.  They  are  interested  in  finding  out  the 
advantages  and  disadvantages  of  graphs  versus  tables.  The  studies  were 
interested  in  determining  how  graphs  affected  decision  quality,  speed,  user 
confidence,  and  the  amount  of  difficulty  the  user  had  in  making  the  decision. 
The  past  studies  seem  to  indicate  that  the  task  to  be  accomplished  has  an 
effect  on  the  mode  of  presentation  that  should  be  used  (1:1 349,3  475). 

Decision  Quality,  Speed,  Confidence  and  Difficulty.  The  results  of  the 
prior  investigations  have  been  inconclusive.  In  their  study,  Benbasat  and 
Dexter  found  no  difference  between  tabular  and  graphical  reports  in  terms 
of  the  quality  of  decision  made.  In  her  research,  DeSanctis  found  seven 
studies  that  favored  graphs,  twelve  that  favored  tables,  and  ten  that  showed 
no  difference  between  the  two  in  terms  of  accuracy,  preference,  problem 
comprehension,  and  decision  quality  (6:470).  No  studies  have  shown  that 
graphics  lead  to  better  confidence  or  a  lower  degree  of  difficulty  in  decision 
making,  or  better  recall  of  the  information  presented  (6:475). 

The  research  also  suggests  that  a  combination  of  the  two  modes  might 
produce  the  best  results.  Benbaset  and  Dexter  conducted  a  study  in  which 
they  evaluated  the  following  forms  of  information  presentation: 


1.  Tabular 


2.  Graphical 

3-  Combined  tabular-graphical  (3:59) 

Their  findings  shoved  that  the  combined  group  significantly  outperformed 
the  graphical  group  in  terms  of  decision  quality.  The  combined  group  also 
did  better  than  the  tabular  group  but  it  was  not  a  significant  difference.  In 
terms  of  decision  speed,  the  tabular  group  took  more  time  than  both  the 
graphical  and  combined  group  (3:70-74).  These  results  seem  to  show  that 
graphs  can  lead  to  a  faster  decision,  while  tabular  data  will  lead  to  a  more 
accurate  decision.  Their  study  also  suggests  that  the  issue  may  not  be  to 
replace  tabular  data  with  graphical  data,  but  use  the  two  together  to 
improve  decision  quality  and  speed. 

Task.  The  past  research  has  been  inconclusive  for  computer  graphics  as  a 
decision  aid.  The  fact  that  some  research  has  shown  graphics  to  be  better 
than  tables  indicates  that  in  some  cases  there  may  be  an  advantage  for 
graphics.  The  problem  is  to  identify  those  conditions  in  which  graphs 
outperform  tables  as  a  mode  of  presentation  (6:475).  Zmud,  Blocher,  and 
Moffie  showed  through  their  research  that  decision  making  with  tabular  and 
graphic  reports  varied  with  task  complexity.  Graphics  were  best  when  the 
complexity  was  low,  and  tables  were  best  when  complexity  was  high  (6:475). 
Benbasat  and  Dexter  feel  that  graphical  and  tabular  reports  need  to  be 
evaluated  in  a  variety  of  settings  to  identify  areas  where  one  has  a 
comparable  advantage  over  the  other  (4:1349).  Ghani  feels  that 
performance  on  a  task  is  contingent  on  the  fit  between  the  task 
characteristics  and  the  mode  of  information  presentation.  The  two  need  to 
reinforce  each  other.  He  has  found  that  tabular  representation  is  best  when 
specific  data  values  are  required,  and  that  graphical  representation  is  best 
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when  relationships  between  data  values  need  to  be  examined  (3:67,  13  4). 
Dickson,  DeSanctis,  and  McBride  conducted  a  study  where  they  compared 
graphical  and  tabular  reports  in  a  number  or  different  situations  (8:41). 
From  their  study,  they  came  up  with  the  following  recommendations: 

1.  Tabular  information  is  best  when  the  task  calls  for  accurate 
interpretation  of  data  value. 

2.  Graphical  representation  is  a  good  choice  when  the  user  needs  to 
see  a  time  dependent  pattern  in  a  large  amount  of  data.  In  other 
words,  graphs  are  good  for  detecting  trends  in  a  large  amount  of  data. 

3-  Either  graphs  are  tables  may  be  used  when  presenting  small 
amounts  of  information  to  an  audience. 

4.  Graphs  are  better  when  presenting  an  audience  with  a  large 
amount  of  information  with  the  purpose  of  having  them  recall  some 
facts  about  the  information  that  was  presented  (8:46). 

The  studies  mentioned  above  indicate  that  the  type  of  task  being  performed 
should  be  considered  when  choosing  a  presentation  format. 

Other  Features.  DeSanctis  has  identified  graph  features  which  could 
enhance  a  manager  s  ability  to  use  graphs  as  decision  support  aids.  These 
features  include:  color,  realism,  and  simplicity  (6:470).  Studies  have  shown 
that  color  graphics  are  preferred  over  achromatic  ones,  and  that  the 
attention  level  of  the  user  is  raised  when  color  is  present.  However,  these 
studies  have  not  shown  color  to  improve  task  performance  or  graph 
comprehension  (6:471).  Realism  has  been  shown  to  raise  the  attention  level 
and  increase  the  retention  rate  among  users.  But  the  studies  do  no  show 
that  realism  aids  comprehension.  In  fact,  the  research  suggests  realistic 
visuals  may  detract  the  user  from  identifying  relevant  pieces  of  information 


(6:471).  Studies  have  also  shown  task  performance  increases  as  the 
simplicity  of  the  graphs  increase.  The  simpler  the  graph,  the  easier  it  is  to 
understand.  None  of  these  factors  (color,  realism,  and  simplicity)  have  been 
examined  to  see  if  they  have  an  effect  on  decision  quality  and  speed  (6:471). 

Individual  Differences 

Another  area  where  research  has  been  done  is  in  the  field  of  individual 
differences.  Individual  differences  refer  to  how  an  individual  processes 
information  (17:966).  Research  in  this  area  studies  how  individual 
differences  may  cause  one  individual  to  be  able  to  use  graphics  more  easily 
than  another  because  of  the  way  their  brain  processes  the  information  that 
is  presented  to  them  (17.966).  The  individual  differences  can  be  broken  into 
three  classes:  cognitive  style,  personality  and  demographics/situational 
(17:967). 

Cognitive  Style.  Cognitive  style  has  to  do  with  the  characteristics  of 
perceptual  and  thinking  behavior  in  an  individual.  The  most  frequent 
cognitive  style  studied  in  graphic  research  has  been  field 
dependence/independence.  Field  dependence/independence  is  defined  as 
follows: 

"In  a  field  dependent  mode  of  perceiving,  perception  is  dominated  by 
the  overall  organization  of  the  field;  there  is  relative  inability  to 
perceive  parts  of  a  field  as  discrete.  This  global  quality  is  indicative  of 
limited  differentiation.  Conversely,  a  field  independent  style  of 
perceiving,  in  which  parts  of  a  field  are  experienced  as  discrete  from 
organized  background,  rather  than  fused  with  it,  is  a  relatively 
differentiated  way  of  functioning"  (2:447). 

Field  independent  individuals  are  said  to  be  more  analytical  than  field 
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dependent  ones.  Several  studies  have  shown  that  field  independent 
individuals  are  better  able  to  extract  information  from  a  graph  than  field 
dependent  individuals  (1 1:795,17:969).  Field  dependents  find  it  difficult  to 
take  apart  a  complex  figure;  therefore,  they  prefer  disaggregated  data.  Field 
independents  performed  equally  well  with  disaggregated  data  and 
aggregated  data  (2:447). 

Other  cognitive  styles  that  might  affect  an  individual's  ability  to  interpret 
computer  graphics  are  simple/complex  and  systematic/heuristic. 
Simple/complex  pertains  to  structural  characteristics  of  thinking  and 
perception.  It  has  three  distinct  properties: 

1.  Differentiation-the  number  of  elements  sought  and  assimilated 

in  cognition; 

2.  Discrimination-the  assigning  of  slightly  varying  stimuli  to  the  same 

or  different  categories; 

3.  Integration-the  number  and  completeness  of  rules  used  in  cognition 

(17:967). 

Systematic/heuristic  shows  whether  or  not  an  individual  uses  abstract 
models  and  systematic  processes  in  cognition,  or  relies  on  common  sense, 
experience,  and  the  practicalities  of  the  situation  (17:967). 

Personality.  Personality  refers  to  the  cognitive  and  affective  structures 
maintained  by  individuals  to  make  adjustments  to  events,  people  and 
situations.  Ambiguity,  tolerance,  extroversion/introversion,  and  anxiety 
level  are  types  of  personality  variables  (17:967).  Currently,  no  conclusive 
results  are  available  relating  personality  to  the  user's  ability  to  use  one 
mode  of  presentation  over  another. 


.  Demographic/situational  variables  cover  a  wide  range  of 


personal  characteristics.  Variables  in  this  area  include  such  things  as  age, 
sex.  experience,  education,  intellectual  ability,  etc.  An  important  situational 
variable  is  the  amount  of  stress  the  decision  maker  is  under.  Studies  have 
shown  that  stress  effects  the  decision  maker  s  ability  to  obtain  and  process 
information  ( 14:643) 

A  demographic  variable  that  has  been  studied  is  experience.  Instead  of 
trying  to  tie  confidence  to  the  mode  of  presentation  used,  the  past  research 
has  found  that  it  is  related  to  the  amount  of  experience  an  individual  has  in 
the  situation  under  study  (18:190).  Experience  is  directly  related  to  the 
amount  of  confidence  an  individual  has  in  the  decision  made.  In  addition  to 
confidence,  experience  is  has  also  been  found  to  be  related  to  the  perceived 
amount  of  difficulty  the  individual  had  in  making  the  decision.  Experience  is 
inversely  related  to  the  the  amount  of  difficulty  the  individual  has  in  making 
the  decision  ( 17:970). 

Some  studies  have  shown  that  individuals  with  training  in  computer 
graphics  are  better  able  get  information  from  them  (6:472).  DeSanctis  and 
Jarvenpaa  found  that  initially  graphical  representation  provided  no 
advantage  over  tabular  form.  However,  after  being  repeatedly  exposed  to 
graphical  representation,  a  significant  difference  was  found  in  the 
performance  of  individuals  using  graphical  information  over  those  using 
tabular  information  when  making  financial  forecasts  (7:140).  This  suggests 
that  a  learning  process  needs  to  take  place  before  a  manager  is  able  to  use 
graphs  effectively. 

Some  researchers  feel  that  graphical  information  is  processed  in  the  right 
side  of  the  brain.  They  feel  this  way  because  the  right  side  is  responsible  for 
imagination,  receptivity,  and  is  more  concrete  in  its  processing  than  the  left 


side.  If  this  theory  is  correct,  people  with  strong  right-brain  dominance 
should  be  able  to  use  graphics  better  than  those  with  left-brain  dominance 
(6:479). 

Future  Research 

Although  research  has  been  done,  the  usefulness  of  graphics  to  a  decision 
maker  is  still  unknown.  The  past  research  is  inconclusive,  and  has  produced 
mixed  results.  It  has,  however,  identified  several  important  variables  for 
future  research.  The  success  of  an  information  system  is  measured  in  terms 
of  its  effectiveness  in  managerial  decision  making.  Two  important  aspects  of 
decision  making  are  the  quality  of  the  decision  and  the  amount  of  time 
required  to  make  the  decision.  Therefore,  these  are  two  important 
dependent  variables  for  future  research.  Other  dependent  variables  that  are 
important  are.  interpretation  accuracy,  problem  comprehension,  task 
performance,  speed  of  comprehension,  memory  for  information,  and  viewer 
preference  (6:467-468).  Some  of  ihe  independent  variables  that  need  to  be 
looked  at  are:  graph  characteristics,  user  demographics,  decision  content, 
and  cognitive  aspects  (6:480). 

One  of  the  more  important  independent  variables  identified  by  past 
research  is  that  of  field  dependence/independence.  Tests  have  shown  that 
field  independents  outperform  field  dependents  when  a  graphical 
representation  of  information  is  used  ( 1 1:795,17:969).  Because  of  these  past 
results,  it  is  important  when  testing  the  mode  of  presentation  to  determine 
whether  the  individual  is  field  dependent  or  independent. 

Other  independent  variables  that  need  to  be  looked  at  when  comparing 
mode  of  presentation  include  the  user's  background.  A  person's  experience 
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or  knowledge  in  the  field  under  study  will  be  a  factor  when  determining 
how  much  confidence  he  has  in  the  decision  he  has  made.  It  will  also  be  a 
contributing  factor  in  determining  the  amount  of  difficulty  he  had  in  making 
the  decision.  Also,  learning  seems  to  be  a  factor  when  using  graphics  as  a 
mode  of  presentation  (7:140).  The  more  a  person  is  exposed  to  graphics  the 
easier  it  is  for  him  to  use  them,  and  to  obtain  information  from  them  (7:140). 
Therefore,  it  is  important  to  know  if  the  subject  has  been  exposed  to 
graphics  as  a  mode  of  information  presentation. 

Another  aspect  to  look  at  is  the  task  to  be  accomplished.  There  is  a  wide 
variety  of  managerial  tasks.  Graphics  may  be  best  suited  for  some,  and 
tabular  data  for  others.  Therefore,  research  needs  to  be  done  that  compares 
graphical  information  to  tabular  information  in  a  wide  variety  of  situations. 
Past  research  has  shown  that  graphics  are  best  when  the  user  is  asked  to 
detect  trends  or  compare  data,  and  tabular  is  best  when  precise  data  values 
are  required  (3:67). 

Through  her  studies,  DeSanctis  developed  a  framework  for  future 
research  in  the  use  of  computer  graphics  as  decision  aids  (see  figure  1) 
(6:480).  The  viability  of  the  framework  is  supported  by  past  research  on 
computer  graphics.  The  framework  shows  that  graph  features,  user 
characteristics,  and  the  decision  content  need  to  be  considered  when 
evaluating  the  usefulness  of  computer  graphics  as  decision  support  aids.  In 
the  area  of  graph  features,  the  information  system  designer  has  to  design 
graphs  which  can  be  easily  interpreted  and  facilitate  quality  decision 
making.  Past  research  has  shown  that  the  user  characteristics  of  graphical 
experience  and  task  knowledge  are  important  variables  (6:481).  The 
framework  shows  how  the  variables  relate  to  each  other. 


Figure  1.  Framework,  for  Research  on  Computer  Graphics  (6:460) 


Computer  graphics  are  becoming  more  widely  used  in  management 
information  systems.  Proponents  claim  graphical  display  of  information  will 
improve  decision  speed  and  quality  (9:1 5).  But  the  initial  research  done 
does  not  support  the  claim.  The  studies  that  have  been  done  have  had 
mixed  results.  Some  studies  favored  graphs,  others  favored  data  in  a  tabular 
form,  while  others  showed  no  difference  between  the  two.  Future  research 
needs  to  be  done  to  determine  where  graphical  information  is  better  than 
tabular  and  vice-versa.  The  past  research  has  indicated  the  areas  where 
future  research  needs  to  be  done,  before  researchers  can  answer  the 
question  of  which  mode  of  information  presentation  is  best  for  a  given 
situation.  Past  research  has  also  identified  several  variables  that  have  an 
effect  on  the  manager  s  ability  to  use  graphics.  The  fundamental  issue 
concerning  researchers  is.  Whether  managers  will  be  able  to  interpret 
accurately  the  information  presented  graphically  and  make  quality  decisions 
based  on  that  information  (6  464)? 
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The  hypotheses  that  were  used  in  this  study  can  be  broken  up  into  three 
types,  those  based  on  effectiveness  measures  of  an  information  system,  task 
to  be  aooomplished,  and  individual  differences.  Based  on  the  past  research, 
these  areas  appear  to  be  the  most  important  aspects  to  consider  when 
comparing  the  mode  of  presentation. 

Effectiveness  Hypotheses 

The  sucoess  of  an  information  system  is  measured  in  terms  of  its 
effectiveness  in  the  managerial  decision  making  prooess.  Decision  speed  and 
quality  are  two  ways  to  measure  the  effectiveness  of  an  information  system 
(6:48 1 ).  For  this  reason  the  following  hypotheses  were  tested  by  this  study: 

Hl:  Individuals  using  graphical  information  will  take  the  same 
amount  of  time  to  evaluate  the  contractor  s  performance  as  the 
individuals  using  tabular  information. 

H2:  Individuals  using  graphical  information  will  perform  as  well  as 
individuals  using  tabular  information  in  terms  of  questions 
answered  correctly. 

These  studies  have  also  concluded  that  the  mode  of  presentation  has  no 
effect  on  the  amount  of  confidence  the  individual  has  in  the  decisions  he 
made,  or  on  how  much  difficulty  the  individual  had  in  making  that 
decisions.  The  following  hypothesis  were  tested  to  validate  the  above 
conclusions: 

H3:  The  amount  of  confidence  an  individual  has  in  the  decision  that 
was  made  will  not  be  related  to  the  mode  of  presentation. 


H4:  The  degree  of  difficulty  an  individual  had  in  making  the  decision 

will  not  be  related  to  tne  mode  of  presentation  used. 

Task  Dependent  Hypotheses 

The  past  research  has  been  inconclusive  in  determining  which  mode  of 

presentation  leads  to  better  decision  making.  Several  studies  have  favored 

graphs,  while  others  have  favored  tables  (4:1349,3:67,6:475).  The  research 

has  indicated  that  the  situation  is  a  determining  factor  when  choosing  a 

mode  of  presentation.  Tabular  information  is  best  for  accuracy  and  graphical 

information  is  best  for  detecting  trends.  In  order  to  test  the  above 

situations,  the  following  two  hypotheses  were  formed: 

H5:  When  asked  for  precise  numbers,  individuals  with  tabular  data 
will  outperform,  in  terms  of  accuracy,  those  individuals  with 
graphical  data. 

H6:  When  asked  to  detect  trends,  or  to  compare  values,  individuals 
with  graphical  data  will  outperform,  in  terms  of  accuracy, 
individuals  with  tabular  data. 

Individual  Differences  Hypotheses 

The  past  research  has  shown  that  an  individual  s  cognitive  style  plays  an 
important  role  in  an  individuals  ability  to  use  graphical  information. 
Previous  studies  have  shown  that  field  independent  individuals  will 
outperform  field  dependent  ones  when  graphical  information  is  used 
(2:447,1 1:795,17:969).  In  addition,  field  dependents  make  better  decisions 
using  tabular  information  than  they  do  with  graphical  information 
(1 1:795,17:969).  For  these  reasons  the  following  hypotheses  were  tested  in 
this  study: 


H7:  Field  independent  individuals  will  score  higher,  in  both  decision 


quality  and  speed,  than  field  dependent  individuals  when 
graphics  are  used. 

H8:  Field  dependent  individuals  will  score  higher,  in  both  decision 
quality  and  speed,  using  tabular  data  than  they  will  using 
graphical  data. 

Another  individual  difference  that  is  important  is  experience  Past 
studies  have  shown  that  the  amount  of  confidence  placed  in  a  decision  is 
positively  related  to  the  number  of  years  of  experience  possessed  by  an 
individual  in  the  situation  being  tested  (18:190).  That  is,  the  more 
experience  an  individual  has  in  a  situation,  the  more  confident  he  is  about 
the  decision  he  made.  Also,  the  years  of  experience  are  inversely  related  to 
the  individuals  perceived  level  of  difficulty  (18:190).  That  is,  an 
inexperienced  individual  will  perceive  the  task  as  being  more  difficult  than 
someone  with  several  years  of  experience.  For  these  reasons  the  following 
two  hypotheses  were  tested  in  this  study: 

H9:  The  degree  of  confidence  an  individual  has  in  the  decision  made 
will  be  positively  related  to  years  of  experience. 

H10:  The  perceived  difficulty  will  be  inversely  related  to  years  of 
experience. 


IV.  Methodol< 


The  methods  used  to  test  the  research  hypotheses  were  experimentation 
and  survey.  Two  test  scenarios  were  given  to  each  member  of  the 
population  One  of  the  scenario  contained  the  information  in  tabular  form 
and  the  other  in  graphical  form.  The  individual  was  asked  to  answer 
questions  based  on  the  information  given.  After  completing  the  scenarios, 
the  individual  was  asked  to  rate  each  of  the  presentation  modes  on 
under standability,  accuracy,  usefulness,  and  satisfaction  using  7  point  Likert 
scales.  Each  individual  was  also  given  a  survey  to  collect  background 
information  (demographics).  Also,  the  Group  Embedded  Figures  Test  was 
given  to  each  individual  to  determine  if  he  was  field  independent  or 
dependent.  The  level  of  data  obtained  by  these  methods  was  ordinal  and 
nominal. 

Task 

The  past  research  has  indicated  that  the  task  to  be  accomplished  is  an 
important  variable  when  comparing  graphical  and  tabular  information.  The 
task  for  this  study  was  to  evaluate  a  contractor’s  performance  using  C/SCSC. 
The  individuals  who  took  the  test  were  asked  to  determine  if  the  contractor 
was  behind  or  ahead  of  schedule,  if  he  was  over  or  under  cost,  if  there  was 
any  relationship  between  the  cost  and  schedule  variance,  and  other 
questions  pertaining  to  the  contractor  s  performance.  The  test  contained 
questions  that  asked  the  individuals  to  detect  trends,  and  questions  that 
required  data  accuracy. 


Population 

Because  of  the  nature  of  the  test,  the  individuals  in  the  sample  population 
had  to  have  some  knowledge  of  C/SCSC.  For  this  reason  the  following 
individuals  were  included  in  the  population: 

1.  Professional  continuing  education  students  attending  the 
Systems  200  program  management  course  at  the  Air  Force  Institute 
or  Technology. 

2.  Air  Force  Institute  Technology  graduate  students  in  the  systems 
management  option  (GSM). 

3.  Program  control  personnel  assigned  to  the  Aeronautical  Systems 
Division  (ASD)  at  Wright  Patterson  A.FD. 

These  individuals  represented  a  cross-section  of  program  managers  and 
cost  analysts  in  the  Air  Force.  However,  the  sample  as  not  random  because 
every  member  of  the  population  did  not  have  a  non-zero  chance  of  being 
chosen.  This  is  because  some  program  managers  had  already  completed  the 
Systems  200  course,  and  some  were  not  yet  qualified  to  take  it,  some  had 
already  completed  their  masters  degree,  and  others  have  not  yet  attended, 
and  some  are  assigned  to  other  organizations  at  other  Air  Force  Bases. 
However,  the  way  the  individuals  were  placed  into  the  two  groups  was 
random. 

Procedure 

The  following  steps  must  be  completed  to  test  the  research  hypotheses: 

1.  Two  test  scenarios  were  designed  to  test  the  manager’s  ability  to 
use  graphics  and  tabular  information  to  evaluate  the  performance  of  a 
contractor.  The  test  included  questions  that  asked  the  subject  to 
detect  trends  in  the  data,  and  questions  that  called  for  precise 
numbers.  Included  after  each  scenario,  were  two  Likert  scales  to 


determine  how  difficult  it  was  for  the  individual  to  make  the  decision 
and  the  degree  of  confidence  the  individual  had  in  the  decision  made. 


2.  The  survey  to  obtain  the  demographics  was  designed  and 
validated. 


3.  Test  times  were  set  up  with  the  individuals  as  follows: 

1.  The  class  schedule  for  the  Systems  200  class  was  obtained, 
and  a  test  time  was  set  up  with  the  instructor.  The  test  was 
given  to  these  students  after  they  had  their  lesson  on  C/SCSC. 

2.  The  section  leader  for  the  GSM's  (class  87s)  was  contacted 
and  a  time  was  arranged  for  the  GSM's  to  take  the  test. 

3.  Individuals  at  ASD  were  called  and  times  were  set  up  to  go 
to  their  office  and  administer  the  test. 


4.  The  Group  Embedded  Figures  Test  was  administered  to  determine 
if  the  individual  was  field  independent  or  dependent.  The  test  was 
administered  and  scored  according  to  the  procedures  outlined  in  the 
Manual  for  the  Embedded  Figures  Tests  ( 1 6:27). 


5.  The  test  scenarios  were  given  to  each  individual.  The  test  scenarios 
were  given  to  individuals  and  groups.  The  group  size  varied  between 
1  and  40.  The  following  steps  were  followed  when  administering  the 
test. 


1.  Each  individual  was  given  a  sheet  containing  key  terms  in 
C/SCSC.  They  were  asked  to  read  the  sheet  and  ask  questions 
on  terms  they  did  not  understand.  This  was  done  to  ensure 
that  each  individual  would  understand  the  terms  used  in  the 
report. 


2.  The  subjects  were  randomly  divided  into  two  groups,  A 
and  B.  Both  groups  were  be  given  two  test  scenarios,  in  the 
first  scenario  Group  A  had  the  information  in  graphical  form 
and  group  B  had  the  information  in  tabular  form.  The  mode 
of  information  presentation  was  switched  in  the  second 
scenario.  Each  individual  was  given  a  copy  of  the  information 
and  the  questions  to  be  answered. 

3.  After  completing  both  scenarios,  each  individual  was  given 
a  questionnaire  asking  him  to  compare  the  two  methods  of 
information  presentation  in  terms  of  accuracy, 
understandability,  usefulness,  satisfaction,  and  preference. 

4.  In  order  to  obtain  the  time  it  takes  an  individual  to 
complete  a  section  of  the  test,  the  individuals  were  timed 
using  a  stop  watch..  When  an  individual  finished  a  test 
scenario,  he  was  asked  to  raise  his  hand,  and  the  time  it  took 
him  to  complete  the  test  was  recorded  on  his  test  paper. 

3.  Each  individual  had  a  maximum  of  20  minutes  to  complete 
each  scenario. 

6.  After  completing  the  test  scenarios,  each  individual  was  given  a 
questionnaire  to  find  out  about  his/her  background.  What  kind  of  job 
experience  they  have,  if  they  have  used  graphics  before,  etc. 


7.  To  test  the  research  hypotheses,  the  data  collected  was  analyzed 
using  SPSSX  multiple  regression/correlation,  paired  t  tests,  and 
ANOVA  packages.  The  type  of  test  used  depended  on  the  hypothesis 
being  tested. 

These  parametric  tests  could  be  used  because  the  population  size 
is  greater  than  30,  and  the  central  limit  theorem  can  be  invoked.  The 


22 


central  limits  theorem  states  that  if  n  is  large  than  the  mean  of  the 
population  approaches  a  normal  distribution  and  a  parametric  test 
may  be  used. 

The  test  procedure  used  to  test  hypothesis  one  and  two  is  as 
follows: 

Ho  4i-42 
Ha:  4l"*42 

Where  u  is  mean  of  the  sample. 

To  test  these  hypotheses  the  oneway  analysis  of  variance  command 
will  be  used.  This  test  procedure  will  give  us  an  F  value,  Ft.  The  null 
hypothesis,  Hq,  will  only  be  rejected  if  Ft  >  Fq  05.  or  if  the  probability 
of  Ft  <  0.05. 

The  test  procedure  used  to  test  hypotheses  five  through  eight  is  as 
follows: 

H0:  m>U2 
Ha:  4i<42 

Where  u  is  mean  of  the  sample. 

Again  the  oneway  analysis  of  variance  command  will  be  used  to  test 
the  hypotheses.  For  these  hypotheses,  the  null  hypotheses  was 
rejected  only  if  Ft  <  Fq  05.  or  the  probability  of  Ft  >  0.05. 

To  test  hypotheses  three  and  four,  the  paired  t  test  was  used.  This 
statistical  test  was  used  because  individuals  evaluated  the  contractor 
using  both  tabular  and  graphical  information.  They  rated  their 
difficulty  and  confidence  in  their  evaluation  for  both  forms  of 
information  portrayal.  This  data  is  paired  because  two  observations 


were  obtained  from  one  individual.  The  test  procedure  is  as  follows: 


H0:  Uj-  U2  =  0 
Ha.  Pi -42*° 

Where  p  is  the  mean  of  the  sample  population. 

The  paired  t  test  will  return  a  t  value,  tt.  The  null  hypothesis  was 
rejected  if  tt  >  tQ  05.  or  the  probability  of  tt  <  0.05.  This  same 
procedure  was  used  to  determine  if  there  was  any  difference  in  the 
ratings  (usefulness,  accuracy,  difficulty,  and  satisfaction)  for  each 
report. 

Regression  analysis  was  used  to  test  hypotheses  nine  and  ten. 
These  hypotheses  are  trying  to  see  if  there  is  any  relationship 
between  the  years  erf  experience  and  the  level  of  confidence  and  years 
of  experience  and  degree  of  difficulty  an  individual  has.  Regression 
analysis  will  return  two  values  that  are  important  to  look  at.  The  F 
value,  Ft  shows  if  they  is  a  relationship  between  the  variables.  If  Ft  > 
Fq  05.  or  the  probability  of  Ft  <  0.05,  then  there  is  a  relationship 
between  the  variables.  The  other  value  returned  by  the  regression 
analysis  is  R2,  or  the  coefficient  of  determination.  This  value  has  a 
range  of  0  to  1.  A  value  of  0  indicates  that  the  relationship  is  not 
linear.  The  closer  the  value  is  to  one,  the  stronger  the  linear 
relationship  is  between  the  two  variable. 

A  pearson  correlation  procedure  was  done  on  all  the  independent 
and  dependent  variables  examined  in  the  study,  to  determine  if  there 
was  any  interactions  between  the  variables.  The  pearson  correlation 
will  return  an  r  value.  The  absolute  value  of  r  tells  if  a  linear 


relationship  exists  between  two  variables.  A  value  of  zero  means  that 
there  is  no  linear  relationship  between  the  variables,  while  a  value  of 
one  indicates  a  strong  linear  relationship.  A  negative  r  value  means 
that  the  two  variables  are  inversely  related  to  each  other,  and  a 
positive  value  means  that  they  are  directly  related. 

All  hypotheses  used  an  alpha  level,  or  level  of  significance,  of  0.05 


V.  Analysis  and  Results 


The  purpose  of  this  study  is  to  determine  which  mode  of  presentation, 
graphs  or  tables,  is  more  effective  in  aiding  program  managers  in  their 
evaluation  of  a  contractor  s  performance  when  C/SCSC  is  used.  Effectiveness  in 
this  study  was  measured  in  terms  of  decision  quality,  speed,  confidence,  and 
difficulty.  In  order  to  determine  which  mode  is  most  effective,  the  research 
hypotheses  had  to  be  tested  and  analyzed.  This  section  discusses  the  analysis 
of  the  hypotheses  and  the  results  that  were  obtained. 


Population 


Seventy-seven  individuals,  60  military  and  17  civilians,  participated  in  this 
study.  Tables  1  thru  IV  provide  demographical  information  on  the  sample. 
Table  1  shows  the  average  age  of  the  individuals,  the  average  number  of  years 
worked  for  the  Air  Force,  and  the  number  of  years  experience  with  C/SCSC. 
The  average  individual  was  30  years  old,  has  been  in  the  Air  Force  for  seven 
years,  and  has  less  than  1  year  of  C/SCSC  experience. 

Table  II  shows  that  74%  of  the  population  tested  was  male.  Tables  Ill  and 
IV  show  how  the  population  was  distributed  among  different  Air  Force 
Specialty  Codes  and  ranks/grades.  Fifty-eight  percent  of  the  individuals 
tested  were  in  the  program  management  career  field,  and  half  of  the 
individuals  were  Lieutenants.  The  majority  of  civilians  tested  were  in  the  30 1 
career  field.  The  grade  spread  was  from  2Lt  to  LtCol  for  the  military,  and 
from  a  GS-7  to  a  GM- 1 3  for  the  civilians. 

These  statistics  show  that  the  majority  of  the  individuals  tested  are 
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involved  in  the  acquisition  process  for  the  Air  Force.  These  individuals  will  be 
involved  in  evaluating  contractor  s  performance,  and  it  is  important  to 
determine  the  best  way  to  present  the  information  to  them.  These  individuals 
represent  the  individuals  who  will  be  making  the  decisions  on  the  major 
weapon  systems  in  the  future. 


Table  1  -  Age,  Years  in  Air  Force,  and  C/SCSC  Experience 


Mean 

Standard 

Deviation 

Range 

Age 

30.312 

7.696 

22-58 

Years  in 

Air  Force 

7.227 

7.684 

1-36 

C/SCSC 

Experience 

0.766 

2.006 

0-15 

Table  II  -  Gender 


Count 

Percentage 

Male 

57 

74.0 

Female 

20 

26.0 

Total 


77 


100.00 


Table  Ill  -  Breakout  by  Air  Force  Specialty  codes  (AFSC) 


AFSC 

Frequency 

Percent 

Military 

26zi 

2 

2.6 

27n 

45 

58.4 

28xx 

4 

52 

49ix 

4 

5.2 

67xx 

4 

52 

70xx 

1 

1.3 

Civilian 

501 

14 

18.2 

345 

3 

3.9 

Total 

77 

100.00 

Effectiveness 


Analysis 


HYPOTHESIS  ONE:  Individuals  using  graphical  information  will  take 
the  same  amount  of  time  to  evaluate  the  contractor's 
performance  as  the  individuals  using  tabular  information. 

The  time  here  refers  to  the  total  amount  of  time  it  took  an  individual  to 
complete  all  three  parts,  evaluation,  accuracy,  and  trend  detection,  of  the  test 
scenario.  The  time  was  recorded  in  seconds. 
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Table  IV  -  Breakout  by  Rank/Grade 


Rank/ 

Grade 

Count 

Percent 

Military 

2Lt 

16 

20.8 

lLt 

23 

29.9 

Capt 

14 

18.2 

Maj 

6 

7.8 

LtCol 

1 

1.3 

Civilian 

GS-7 

2 

2.6 

GS-9 

3 

3.9 

GS-11 

6 

7.8 

GS-12 

2 

2.6 

GS-13 

2 

2.6 

GS-14 

1 

1.3 

GM-15 

1 

1.3 

Total 

77 

100.00 

1 


The  oneway  analysis  of  variance  command  in  SPSSX  was  used  to  compare 
the  time  taken  to  complete  the  evaluation  with  graphical  information  and 
the  time  taken  with  tabular  information.  The  mean  time  required  by 
individuals  using  graphical  information  was  416  seconds,  and  the  mean  for 
tabular  information  was  slightly  less  at  401.  However,  this  difference  is  not 
significant  at  the  0.05  level.  Therefore,  hypothesis  one  can  not  be  rejected. 
The  results  of  the  analysis  are  summarized  in  Tables  V  and  VI. 
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Table  V  -  Time  To  Complete  Test  (Time  in  seconds) 


Mode  of 

Standard 

Presentation 

Mean 

Deviation 

Count 

Min 

Max 

Tabular 

401.4416 

132.9685 

77 

145 

790 

Graphical 

416.1299 

122.4317 

77 

154 

691 

Total 

413.7208 

127.6032 

154 

145 

790 

Table  VI  -  Oneway  Analysis  of  Variance  -  Time  by  Mode  of  Presentation 


Degrees  of 

Sum  of 

Mean 

F 

F 

Source 

Freedom 

Squares 

Squares 

Ratio 

Prob 

Between 

Groups 

Within 

1 

8306.2403 

8306.2403 

0.5085 

0.4769 

Groups 

152 

2482929.688 

163350637 

Total 

153 

2491235929 

HYPOTHESIS  TWO;  Individuals  using  graphical  information  will 
evaluate  the  contractors  performance,  in  terms  of  questions 
answered  correctly,  as  well  as  individuals  using  tabular 
information. 


Scores  for  evaluation  were  based  on  the  number  of  questions  answered 
correctly  on  the  evaluation  part  of  the  test.  Evaluation  involves  determining 
whether  the  contractor  is  ahead  or  behind  schedule  and  under  or  over  cost. 
These  are  questions  1  through  4  on  the  tabular  test,  and  questions  1,  2,  8, 


and  9  on  the  graphical  test  (See  Appendix  A).  The  maximum  possible  score 
is  8.  The  oneway  analysis  of  variance  command  was  used  again  to 
determine  if  there  is  a  difference  in  means. 

The  analysis  showed  that  individuals  using  graphical  information 
performed  better  at  evaluating  the  contractor*  s  performance.  The  mean  for 
individuals  using  graphs  was  7.7 143,  and  7.2857  for  individuals  using  tables. 
This  difference  is  statistically  significant  at  the  the  0.05  level.  This  result 
leads  leads  to  the  rejection  of  hypothesis  two.  These  results,  in  this  case, 
indicate  that  graphical  information  was  better  for  determining  whether  a 
contractor  is  ahead  or  behind  schedule,  and  over  or  under  cost.  Tables  VII 
and  VIII  summarize  the  results  of  the  analysis. 


Table  VU  -  Contract  Evaluation  Test  Scores  by  Mode  of  Presentation 


Mode  of 

Standard 

Presentation 

Mean 

Deviation 

Count 

Min 

Max 

Tabular 

7.2857 

1.7157 

77 

2 

8 

Graphical 

7.7  M3 

0.6460 

77 

5 

8 

Total 

7.5000 

1.3098 

154 

2 

8 
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Table  VI II  -  Oneway  Analysis  of  Variance  -  Contract  Evaluation  Test  Scores 

by  Mode  of  Presentation 


Degrees  of 

Sum  of 

Mean 

F 

F 

Source 

Freedom 

Squares 

Squares 

Ratio 

Prob 

Between 

Groups 

1 

7.0714 

7.0714 

4.2081* 

0.0419 

Within 

Groups 

152 

255-4286 

1.6805 

Total 

153 

262.500 

‘Significant  F  value 


HYPOTHESIS  THREE:  The  amount  of  confidence  an  individual  has  in 
the  decision  that  was  made  will  not  be  related  to  the  mode  of 
presentation. 

After  completing  a  test  the  individual  was  asked  to  rate  the  amount  of 
confidence  he  had  in  his  evaluation  of  the  contractor.  A  seven  point  Likert 
scale  was  used  to  do  this.  A  rating  of  seven  represents  the  highest  degree  of 
confidence,  and  a  rating  of  one  the  lowest. 

The  SPSSX  paired  t-test  was  used  to  determine  if  the  mode  of  presentation 
had  any  effect  on  the  perceived  amount  of  confidence  an  individual 
possessed  in  the  decision  that  was  made.  The  mean  amount  of  confidence 
displayed  for  individuals  using  tables  was  5-4935  The  mean  for  graphical 
information  was  slightly  less  at  5 .4286.  However,  this  difference  is  not 
significant  at  the  0.05  level.  Therefore,  the  hypothesis  can  not  be  rejected. 
This  result  supports  previous  studies  in  showing  that  the  mode  of 


presentation  has  no  effect  on  the  amount  of  confidence  possessed  by  an 
individual  (6:475).  Table  IX  summarizes  the  results  of  the  analysis. 

HYPOTHESIS  FOUR:  The  degree  of  difficulty  an  individual  had  in 
making  a  decision  will  not  be  related  to  the  mode  of 
presentation  used. 

Along  with  rating  the  confidence  they  had  in  their  evaluation,  individuals 
were  also  asked  to  rate  the  amount  of  difficulty  they  had  in  evaluating  the 
contractor  s  performance.  A  rating  of  seven  in  this  case,  means  the 
individual  had  a  great  deal  of  difficulty,  and  a  rating  of  one  means  the 
individual  had  no  difficulty  in  his  evaluation. 

Like  hypothesis  three,  the  SPSSX  paired  t  test  was  used  to  analyze  the 
results.  The  analysis  showed  that  individuals  using  graphical  information 
perceived  themselves  as  having  slightly  less  difficulty,  2.7792,  than  those 
using  tabular  information,  2.9221.  This  difference  is  slightly  larger  than  the 
difference  in  confidence  means,  but  again  it  is  not  significant  at  the  0.05 
level.  This  means  that  hypothesis  four  can  not  be  rejected.  These  findings 
again  confirm  the  findings  of  previous  studies  that  showed  that  difficulty 
had  nothing  to  do  with  the  form  of  information  presentation  (6:475).  A 
summary  of  this  analysis  is  found  in  Table  IX. 
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Table  IX  -  Confidence  and  Difficulty  vs  Mode  of  Presentation  (Paired  T-test) 


Standard  Standard 

Mean  Mean  Dlf-  Deviation  Deviation  T  T 
Tables  Graphs  ference  Graphs  Tables  Value  Prob 

Confidence 

5.4935  5.4286  0.0649  1.096  1.057  0.59  0.556 

Difficulty 

2.9221  2.7792  0.1492  1.355  1.199  1.17  0  246 

N=77  for  all  cases 


Task  Dependent  Hypotheses  Analysis 

HYPOTHESIS  FIVE:  When  asked  for  precise  numbers,  individuals 
with  tabular  information  will  outperform,  in  terms  of  accuracy, 
individuals  with  graphical  information  in  terms  of  accuracy. 

To  test  this  hypothesis,  five  questions,  questions  5  through  9  on  the 
tabular  test,  and  questions  3  through  7  on  the  graphical  test  (See  Appendix 
A)  were  on  each  test  asking  the  individual  to  come  up  with  specific  numbers. 
The  number  the  individual  put  down  was  marked  correct  IT  it  was  within  5% 
of  the  actual  number  (plus  or  minus).  The  maximum  possible  score  was  5. 

The  one  way  analysis  of  variance  command  was  used  to  determine  if  the 
mean  for  the  tabular  group  was  higher  than  the  mean  for  the  graphical 
group.  The  results  showed  that  the  mean  for  the  tabular  group,  4.948 1,  was 


significantly  higher  than  that  of  the  graphical  group,  3-5325,  at  the  0.05  level 
of  significance.  This  means  that  hypothesis  four  can  not  be  rejected.  These 
results  are  consistent  with  those  of  studies  done  in  the  past  (3:70,  8:41).  In 
order  do  obtain  exact  values  from  a  graph  a  certain  amount  of  estimation  is 
involved  depending  on  the  scale  of  the  graph.  If  the  accuracy  desired  is 
beyond  the  scales  graduations,  then  the  individual  is  forced  to  estimate 
more.  The  test  results  further  support  this  rather  obvious  fact.  The  results 
showed  that  it  got  even  harder  to  estimate  the  number  as  the  number  got 
smaller.  Individuals  were  fairly  accurate  in  estimating  the  number  as  long 
as  it  was  greater  than  one,  but  had  difficulty  when  it  was  less  than  one.  This 
shows  that  if  precise  numbers  are  required,  it  is  best  to  give  the  information 
in  tabular  form.  The  results  of  this  analysis  can  be  found  in  Tables  X  and  XI. 


Table  X  -  Accuracy  by  Mode  of  Presentation 


Mode  of 

Presentation 

Mean 

Standard 

Deviation 

Count 

Min 

Max 

Tabular 

4.9481 

0.2761 

77 

3 

5 

Graphical 

3.5325 

0.7179 

77 

2 

5 

Total 


4.2403 


0.8934  154 


2 


5 


Table  XI  -  Oneway  Analysis  of  Variance  -  Accuracy  by  Mode  of  Presentation 


Degrees  of 

Sum  of 

Mean 

F 

F 

Source 

Freedom 

Squares 

Squares 

Ratio 

Prob 

Between 

Groups 

1 

77.1494 

77.1494 

260.8 192‘ 

0.0000 

Within 

Groups 

152 

44.9610 

0.2958 

Total 

153 

122.1104 

‘Significant  F  value 


HYPOTHESIS  SIX:  When  asked  to  detect  trends,  or  to  compare 
values,  individuals  with  graphical  information  will  outperform, 
in  terms  of  questions  answered  correctly,  individuals  with 
tabular  data  in  terms  of  questions  answered  correctly. 

Five  questions  were  included  on  the  tests,  questions  10  thru  14  on  both 
exams,  asking  the  individual  to  detect  trends  or  compare  values.  This  part  of 
the  exam  was  scored  by  the  number  of  trends  detected  correctly  or  the  right 
comparison  being  made.  The  maximum  possible  score  was  5. 

The  same  procedure  used  to  test  hypothesis  five  was  also  used  to  test 
hypothesis  six.  The  analysis  revealed  that  information  in  graphical  formed 
allowed  the  individual  to  detect  the  trends  better  than  information 
presented  in  tabular  form.  The  mean  for  the  graphical  group  was  4.3506, 
while  the  mean  for  the  tabular  group  was  3.9351.  The  difference  in  these 
means  is  significant  at  the  0.03  level,  and  therefore,  the  hypothesis  is  not 
rejected.  These  findings  support  the  studies  in  the  past  that  have  shown 


that  graphical  information  is  better  for  showing  trends  than  tabular 
information  (3:67,  8:41,  15:4).  The  results  of  the  analysis  are  summarized  in 
Tables  XII  and  XIII. 


Table  XII  -  Trend  Detection  by  Mode  of  Presentation 


Mode  of 

Presentation 

Mean 

Standard 

Deviation 

Count 

Min 

Max 

Tabular 

3.9351 

0.8325 

77 

2 

5 

Graphical 

4.3506 

0.8851 

77 

1 

5 

Total 

4.1429 

0.8841 

154 

1 

5 

Table  XIII  -  Oneway  Analysis  of  Variance  -  Trend  by  Mode  of  Presentation 


Source 

Degrees  of 
Freedom 

Sum  of 
Squares 

Mean 

Squares 

F 

Ratio 

F 

Prob 

Between 

Groups 

1 

6.6494 

6.6494 

9.0074* 

0.0031 

Within 

Groups 

152 

112.2078 

0.7382 

Total 

153 

118.8571 

•Significant  F  value 


Individual  Difference  Hypotheses  Analysis 


HYPOTHESIS  SEVEN:  Field  independent  individual  will  score  higher, 
in  terms  of  contract  evaluation,  time,  trend  detection,  and 
accuracy,  than  field  dependent  individuals  when  graphics  are 
used. 

Before  this  hypothesis  can  be  analyzed,  the  sample  population  needs  to 
be  broken  up  into  field  independent  and  field  dependent  individuals.  To  do 
this  the  group  embedded  figures  test  was  administered  and  scored  according 
to  the  procedures  in  the  Manual  for  the  Embedded  Figures  Tests  (16:27). 
These  scores  were  analyzed  and  the  results  are  in  Table  XIV.  The  mean  for 
the  test  was  13-81.  Based  on  the  test  mean,  13-81,  individuals  who  scored 
13  or  less  were  classified  as  field  dependent,  and  those  that  scored  14  or 
higher  were  classified  as  field  independent.  This  mean  is  slightly  higher 
than  the  mean  stated  in  the  Manual  for  the  Embedded  Figures  Tests,  12.0, 
(16:28).  However,  it  is  the  same  cutoff  that  Benbaset  and  Dexter,  Lusk,  and 
Docktor  and  Hamilton  used  in  their  studies  (4:1333).  This  cutoff  point 
resulted  in  44  individuals  being  classified  as  field  independent  and  33  as 
field  dependent. 


Table  XIV  -  Group  Embedded  Figures  Test 


Mean:  13.831  Minimum:  4 

Standard 

Deviation:  4.092  Maximum:  18 

Field  Dependent  <»  1 3  Field  Independent  >- 1 4 
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In  order  to  test  the  hypothesis,  Tour  different  analysis  of  variance  tests 
were  conducted  using  the  SPSSX  oneway  command.  The  first  one  tested  to 
aee  if  there  was  any  difference  in  the  amount  of  time  it  took  to  complete  the 
test  for  the  two  groups.  The  analysis  showed  that  the  mean  time  for  field 
independent  group  to  complete  the  test  using  graphical  information  was  less 
than  the  mean  time  for  field  dependent  group,  402.0453  to  434.9091. 
However,  this  difference  is  not  significant  at  the  0.05  level.  Consequently, 
this  result  does  not  support  the  hypothesis. 

The  second  analysis  compared  the  evaluation  scores  for  both  groups.  The 
results  again  showed  that  field  independents  scored  higher  than  field 
dependents,  but  this  time  the  difference  was  significant  at  the  0.05  level. 
This  result  supports  hypothesis  seven. 

The  third  analysis  compared  the  scores  for  the  accuracy  portion  of  the 
test.  Again,  the  mean  for  field  independents,  3.6591,  was  higher  than  the 
mean  for  field  dependents,  3  3636.  Yet,  this  difference  was  not  significant  at 
the  0.05  level. 

The  final  analysis  was  done  to  compare  the  scores  for  trend  detection. 
The  findings  here  showed  that  field  independents  did  significantly  better 
than  the  field  dependents,  4.5227  to  4.1212.  This  supports  the  hypothesis. 

The  results  of  this  analysis  shows  that  field  independents  did  perform 
better  than  field  dependents  when  graphs  were  used  for  the  mode  of 
presentation.  However,  the  difference  was  only  significant  for  the  evaluation 
and  trend  detection  parts  of  the  test.  This  suggests  that  in  some  situations 
field  independent  and  dependents  should  be  able  to  use  graphs  without  any 
significant  difference  in  performance.  The  results  of  these  tests  are 
summarized  in  Table  XV  and  XVI. 
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Table  XV  -  Graphical  Results  by  Field  Dependent/ Independent 


Count 

Mean 

Standard 

Deviation 

Minimum  Maximum 

Time 

Dependent 

33 

434.9091 

127.644 

170 

660 

Independent 

44 

402.0455 

117.8744 

154 

691 

Evaluation 

Dependent 

33 

7.4848 

0.8704 

5 

8 

Independent 

44 

7.8864 

0.3210 

7 

8 

Accuracy 

Dependent 

33 

3.3636 

0.7424 

2 

5 

Independent 

44 

3.6591 

0.6801 

2 

5 

Trend 

Dependent 

33 

4.1212 

1.0535 

1 

5 

Independent 

44 

4.5227 

0.6985 

3 

5 

HYPOTHESIS  EIGHT:  Field  dependent  individuals  vill  score  higher, 
in  terms  of  contract  evaluation,  time,  accuracy,  detection  of 
trends,  using  tabular  data  than  they  vill  using  graphical  data. 

The  method  of  analysis  for  hypothesis  eight  is  the  same  as  hypothesis 
seven,  except  the  mean  comparison  is  done  for  field  dependents  only.  The 
analysis  of  variance  test  was  done  for  the  four  areas,  time,  contractor 
evaluation,  accuracy,  and  trend  detection.  A  summary  of  the  results  can  be 
found  in  table  XVII  and  XVIII. 


Table  XVI  - 

Oneway  Analysis  of  Variance  - 

Graphs  by  Field 

Indepencience/Dependence 

Degrees  of  Sum  of 

Mean 

F 

F 

Source 

Freedom  Squares 

Squares 

Ratio 

Prob 

Time 

Between 

Groups 

Within 

1 

20366.0649 

20366.0649 

1.3652 

0.2463 

Groups 

75 

1118836.636 

14917.8218 

Total 

76 

1139202.701 

Evaluation 

Between 

Groups 

Within 

1 

3.0400 

3.0400 

7.9515* 

0.0061 

Groups 

75 

28.6742 

0.3823 

Total 

76 

31.7143 

Accuracy 

Between 

Groups 

Within 

1 

1.6461 

1.6461 

32902 

0.0737 

Groups 

75 

37.5227 

0.5003 

Total 

76 

39.1688 

Trend 

Between 

Groups 

Within 

1 

3.0400 

3.0400 

4.0360' 

0.0481 

Groups 

75 

56.4924 

0.7532 

Total 

76 

59.5325 

'Significant  F  value 
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The  results  of  the  analysis  show  that  field  dependents  took  less  time  and 
did  better  on  the  accuracy  part  of  the  test  when  tabular  information  was 


used  as  opposed  to  graphical  information.  However,  the  difference  was 
significant  at  the  0.05  level  only  for  the  accuracy  part.  On  the  evaluation 
and  trend  detection  parts  of  the  test  field  dependents  did  better  when 
graphical  data  was  used,  but  the  difference  was  not  significant.  From  these 
results,  hypothesis  eight  is  only  supported  when  the  questions  involve 
accuracy;  otherwise  it  would  have  to  be  rejected  for  the  other  cases. 


Table  XVII  -  Field  Dependent  Results  by  Mode  of  Presentation 


Count 

Mean 

Standard 

Deviation 

Minimum  Maximum 

Time 

Graphical 

33 

434.9091 

127.644 

170 

660 

Tabular 

33 

4133636 

1453915 

145 

659 

Evaluation 

Graphical 

33 

7.4848 

0.8704 

5 

8 

Tabular 

33 

7.2424 

1.6589 

2 

8 

Accuracy 

Graphical 

33 

3.3636 

0.7424 

2 

5 

Tabular 

33 

4.9091 

0.3844 

3 

5 

Trend 

Graphical 

33 

4.1212 

1.0535 

1 

5 

Tabular 

33 

3.8788 

0.7809 

2 

5 
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Table  XV111  -  Oneway  Analysis  of  Variance  -  Mode  of  presentation  by  Field 

Dependence 


Degrees  of  Sum  of  Mean  F 

Source  Freedom  Squares  Squares  Ratio 

Time 

Between 

Groups  1  7659.4091  7659.4091  0.4092 

Within 

Groups  64  1197816.364  18715.8807 

Total  65  1205475.773 


Evaluation 

Between 

Groups  1 

Within 

Groups  64 

Total  65 


0.9697  0.9697  0.5526 

112.3030  1.7547 

113.2727 


Accuracy 

Between 

Groups  1 

Within 

Groups  64 

Total  65 


39.4091  39.4091  112.7805* 

22.3636  0.3494 

61.7727 


Trend 

Between 

Groups  1 

Within 

Groups  64 

Total  65 


0.9697  0.9697  1.1278 

55.0303  0.8598 

56.0000 


'Significant  F  value 
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F 

Prob 


0.5246 


0.4600 


0.0000 


0.2922 


HYPOTHESIS  NINE:  The  degree  of  confidence  an  individual  has  in  the 
contract  evaluation  will  be  positively  related  to  the  years  of 
experience. 

In  order  to  thoroughly  test  the  hypothesis,  two  different  regression 
analyses  were  done.  The  first  one  compared  total  Air  Force  experience  with 
the  amount  of  confidence,  and  the  second  one  compared  the  amount  of 
C/SCSC  experience  with  the  level  of  confidence.  Amount  of  experience  was 
measured  by  the  number  of  years  the  individual  worked  in  that  area.  The 
first  analysis  showed  that  there  is  an  inverse  relationship  between  the 
amount  of  Air  Force  experience  and  the  individual  s  confidence  level  at  a 
level  of  significance  of  alpha  -  0.05.  However,  the  coefficient  of 
determination,  r^,  is  relatively  small,  which  means  there  is  not  a  strong 
linear  relationship  between  the  two  variables,  experience  and  confidence. 
Also  the  value  of  beta  is  negative,  which  indicates  an  inverse  relationship. 
The  second  analysis  showed  that  there  is  no  relationship  between  the 
number  of  years  erf  C/SCSC  experience  and  level  of  confidence.  The  results  of 
the  two  regression  analyses  do  not  support  hypothesis  nine.  A  summary  of 
these  analyses  can  be  found  in  table  XIX. 

HYPOTHESIS  TEN:  The  individual  s  perceived  level  of  difficulty 
will  be  inversely  related  to  years  of  experience. 

In  order  to  adequately  test  this  hypothesis  two  regression  analyses  were 
also  done.  This  time  the  analysis  involved  comparing  the  years  of  Air  Force 
experience  and  the  years  of  C/SCSC  experience  to  the  amount  of  difficulty 
the  individual  had  in  evaluating  the  contractor's  performance.  The  results  of 
both  regression  analyses  showed  that  they  was  no  relationship  between  the 
amount  of  difficulty  and  the  number  of  years  of  experience  either  in  the  Air 
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Force  or  C/SCSC.  The  results  of  the  analysis  are  summarized  in  table  XX 
Consequently,  this  led  to  rejection  of  hypothesis  ten. 


Table  XIX  -  Confidence  -  Experience  Regression  Analysis 


1 

I 

I 

Standard 

T 

Significance 

Beta 

C/SCSC 

Error  Beta 

Constant 

Value 

of  T 

1 

) 

) 

1 

i 

j 

Experience  -0.042890 

r2  -  0.00642 

Air  Force 

0.043263 

5.493903 

-0.991 

0.3231 

i  * 

i 

i 

i 

Experience  -0.033553 

r2  -  0.05770 

0.010988 

5703536 

-3051 

0.0027 

The  results  of  the  above  regression  analysis  seem  to  show  that  there  is 
another  factor  involved  in  determining  the  the  level  of  confidence  and  the 
degree  of  difficulty  other  than  the  years  of  experience.  The  amount  of 
experience  in  C/SCSC  was  probably  not  a  good  indicator  because  most  of  the 
individual  tested  had  only  been  exposed  to  it  for  under  one  month.  A 
pearson  correlation  was  be  done  to  see  what  other  variables  effect 
confidence  and  difficulty.  This  correlation  analysis  included  all  the 
dependent  and  independent  variables  to  see  what  other  relationships  may 
exist. 
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Table  XX  -  Difficulty  -  Experience  Regression  Analysis 


Standard  T  Significance 

B  Error  B  Constant  Value  of  T 

C/SCSC 

Experience  -0.060717  0.051409  2.897173  -1.181  0.2394 

r2  -  0.00909 
Air  Force 

Experience  0.012975  0.013440  2.756872  0.965  0.3358 

r2  -  0.00609 


Pearson  Correlation 

The  pearson  correlation  analysis  outputs  a  value  called  r.  This  value  ranges 
from  a  -1  to  +1.  The  absolute  value  of  r  is  the  strength  of  the  linear 
relationship  between  two  variables.  A  negative  value  of  r  tells  you  that  an 
inverse  relationship  exists,  while  a  positive  value  indicates  a  positive 
relationship.  The  pearson  correlation  analysis  was  done  using  the 
independent  variables,  such  as  field  independent/dependent  and  experience, 
and  the  dependent  variables,  such  as  time,  evaluation,  accuracy,  trend, 
confidence,  and  difficulty.  The  mode  of  presentation  was  not  included  in  the 
analysis  because  it  has  already  been  compared  to  all  of  the  dependent 
variables.  The  results  of  the  analysis  are  found  in  table  XXI  -  XX1U. 

Table  XXI  shows  the  interactions  between  the  dependent  variables.  The 
results  of  the  pearson  correlation  reveals  that  the  amount  of  time  it  took  an 


individual  to  evaluate  the  contractor's  performance  had  an  effect  on  his 
confidence  and  difficulty.  For  eiample,  the  less  time  he  took,  the  more 
confidence  he  had  in  his  evaluation.  Conversely,  individuals  taking  more 
time  had  more  difficulty  and  less  confidence  in  their  evaluation  of  the 
contract.  The  results  indicate  that  the  amount  of  time  an  individual  took  had 
no  effect  on  how  veil  he  did  in  terms  of  accuracy,  contract  evaluation,  and 
trend  detection.  The  results  also  show  that  the  strongest  relationship  is 
between  confidence  and  difficulty,  which  are  inversely  related  to  each  other. 
That  is,  the  more  confident  an  individual  is,  the  less  difficult  he  perceived 
the  task  to  be.  Other  variables  that  have  a  positive  relationship  with 
confidence  are  the  scores  on  the  accuracy  and  trend  portions  of  the  test.  As 
the  scores  on  these  portions  of  the  test  went  up  the  confidence  level 
increased.  The  scores  on  the  trend  portion  also  have  an  inverse  relationship 
with  the  individual  s  perceived  amount  of  difficulty. 


Table  XXI  -  Interaction  Between  Variables  -  Dependent  Variables 


Table  XXI 1  shows  the  interactions  between  the  dependent  and 
independent  variables.  The  variable  eft  is  the  scores  obtained  by  the 
individual  on  the  group  embedded  figures  test.  This  variable,  eft,  had  an 
effect  on  the  level  of  confidence  and  the  amount  of  difficulty  an  individual 
had.  Individuals  who  scored  higher  on  the  group  embedded  figures  test  had 
less  difficulty  and  more  confidence  in  their  evaluation  than  those  individuals 
who  did  not  score  very  high.  Another,  relationship  that  can  be  detected  is 
the  one  between  C/SCSC  experience  and  the  amount  of  time  it  took  an 
individual  to  complete  the  exam.  Subjects  with  more  C/SCSC  experience 
tended  to  take  less  time  to  complete  their  evaluation.  This  is  not  surprising 
because  the  more  familiar  an  individual  is  with  a  task,  the  less  time  it  should 
take  him  to  complete  it.  As  noted  above  in  the  regression  analysis,  the 
number  of  years  of  Air  Force  experience  has  an  inverse  relationship  with  the 
level  of  confidence. 

Table  XIII  shows  the  relationships  between  the  dependent  variables.  NO 
results  of  any  real  value  can  be  determined  here.  The  only  thing  noticeable 
is  that  the  subjects  with  less  experience,  both  Air  Force  and  C/SCSC,  tended 
to  score  higher  on  the  group  embedded  figures  test. 


|  Table  XXII  -  Interaction  Between  Variables  -  Dependent  -  Independent 

[  Time 

Eval  Acc  Trend 

Conf 

Diff 

j  Eft  -0.1051 

0.0135  0.0833  0.2176 

0.2299 

-0.1918 

(  154) 

(  154)  (  154)  (  154) 

(  154) 

(  154) 

p-  .097 

p=  .434  p=  .138  .003 

p=  .002 

p=  .009 

C/SCSC  -0.2036 

-0.0522  -0.0232  -0.0697 

-0.0802 

-0.0954 

(  154) 

(  154)  (  154)  (  154) 

(  154) 

(  154) 

p-  .006 

p-  .260  p=  .388  p«  .195 

p«  .162 

p=  .120 

AF  Years  -0.0256 

-0.0607  -0.0251  -0.1226 

-0.2402 

0.0781 

(  154) 

(  154)  (  154)  (  154) 

(  154) 

(  154) 

p=  .377 

p=  .227  p=  .378  p=  .065 

p=  .001 

oo 
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Table  XXIII 

-  Interaction  Between  Variables  -  Independent  Variables 

Eft  C/SCSC  AF  Years 

Eft 

* 

1.0000  -0.1744  -0.1702 

(  0)  (  154)  (  154) 

p=  .  p*  .014  p«  .017 

C/SCSC 

f 

» 

1.0000  0.3223 
(  0)  (  154) 

p=  .  p*  .000 

Years 

1.0000 
(  0) 

P-. 
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Subject's  Ratings  of  Report  Attributes 

After  completing  both  test  scenarios  the  subject  was  asked  to  rate  each 
mode  of  presentation  on  usefulness,  accuracy,  difficulty,  and  satisfaction. 
The  results  of  these  ratings  are  displayed  in  Table  XXIV.  For  all  cases  the 
best  rating  is  a  one,  and  the  worst  a  seven.  From  the  table  it  can  be  seen 
that  the  individuals  thought  the  graphical  information  was  more  useful  than 
‘  tabular.  Graphical  information  was  also  rated  less  difficult,  and  the  subjects 
were  more  satisfied  with  it.  The  only  area  that  tabular  information  was 
rated  higher  than  graphical  was  accuracy.  This  is  not  surprising,  because  it 
is  hard  to  get  a  precise  number  off  a  graph  when  the  scale  is  larger  than  the 
desired  accuracy. 

To  determine  if  any  of  these  differences  was  significant  a  paired  t-test 
was  run  using  SPSSX.  The  results  of  this  test  is  in  Table  XXV.  The  test 
showed  that  the  difference  was  significant,  at  the  0.05  level,  in  the  areas  of 
usefulness  and  accuracy.  From  this  result,  it  can  be  concluded  that  graphs 
are  more  useful  while  tables  are  more  accurate. 

Along  with  rating  the  report,  the  subject  were  also  asked  which  format 
they  preferred,  and  which  one  they  would  prefer  if  they  had  to  take  the  test 
again.  Their  responses  were  analyzed  and  the  results  showed  that  61.o%  of 
the  individuals  liked  the  graphical  report  better  than  the  tabular  one.  Also, 
61%  said  they  would  prefer  to  have  the  graphical  format  if  they  were  to  take 
the  test  again.  Table  XXVI  summarizes  all  of  these  results. 


Table  XXIV  -  Attribute  Ratings 


Mean 

Standard 

Deviation 

Min 

Max 

Usefulness 

Graphical 

2.6364 

1.555 

1 

6 

Tabular 

3.2338 

1.661 

1 

7 

Accuracy 

Graphical 

4.1818 

1.412 

2 

7 

Tabular 

1.8961 

1.220 

1 

6 

Difficulty 

Graphical 

2.6753 

1.626 

1 

6 

Tabular 

3.1429 

1393 

1 

6 

Satisfaction 

Graphical 

2.9351 

1.499 

1 

7 

Tabular 

32208 

1.501 

1 

7 

Table  XXV  -Attribute  Ratings  vs  Mode  of  Presentation  (Paired  T-test) 


Mean 

Difference 

Standard 

Error 

Degrees 
of  Freedom 

T 

Value 

Prob 

|  Usefulness 

-0.5974 

0.271 

76 

-2.20 

0.031 

Accuracy 

2.2857 

0.223 

76 

10.27 

0.000 

Difficulty 

-0.4675 

0.240 

76 

-195 

0.055 

Satisfaction 

-0.2857 

0.256 

76 

-1.12 

0.268 

i- 

■  N-77  for  all  cases 

i 

i 

i 

! 

! 

I 
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j  Table  XXVI  -  Preferences 

i 

Count 

Percentage 

Liked  the  Best 

Graphical 

47 

61.0 

Tabular 

30 

39.0 

Total 

77 

100.00 

i 

i 

Prefer  if  had  to  take  the  test  again 

1 

i 

Graphical 

47 

61.0 

» 

i 

Tabular 

30 

39.0 

i 

Total 

77 

100.00 

IUBMAaTV 
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VI.  Conclusions  and  Recommendations 


RESEARCH  QUESTION:  When  using  cost/schedule  control  system 
criteria  to  evaluate  a  contractor  s  performance,  which  mode  of 
presentation,  tabular  or  graphical,  will  be  more  effective  in 
aiding  program  managers  in  terms  of  decision  quality,  speed, 
confidence,  and  difficulty?  Will  graphical  or  tabular 
information  lead  to  a  better  understanding  of  the  situation? 

In  order  to  thoroughly  answer  this  research  question  each  aspect, 
decision  quality,  decision  speed,  confidence,  and  difficulty,  was  eiamined 
separately.  Included  in  decision  quality  are  the  areas  of  contract  evaluation, 
accuracy,  and  trend  detection. 

In  the  area  of  decision  quality,  graphical  information  led  to  better 
performance  in  the  areas  of  contract  evaluation  and  trend  detection,  while 
tabular  data  was  best  for  accuracy.  However,  if  just  field  dependent 
individual's  scores  were  analyzed,  no  significant  difference  was  found  in  the 
areas  of  contract  evaluation  and  trend  detection.  The  scores,  for  field 
dependents,  were  higher  when  graphical  information  was  used,  but  it  was 
not  significantly  better  than  the  scores  when  tabular  information  was  used. 
Determining  whether  or  not  the  contractor  is  ahead  or  behind  schedule,  and 
over  or  under  cost  is  important  in  this  situation,  because  it  will  determine 
how  much  more  analysis  in  needed.  If  the  contractor  is  on  schedule  and 
cost,  no  further  analysis  needs  to  be  done.  If  he  is  over  cost  or  behind 
schedule,  the  situation  needs  to  be  checked  into  further.  Trend  detection  is 
needed,  because  it  is  important  to  see  if  there  is  an  overall  trend  developing 
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in  the  contractor's  performance.  Is  he  falling  further  behind,  or  is  this  just  a 
one  month  slippage?  Accuracy  will  be  become  more  important  later  when 
the  information  is  used  to  estimate  the  final  cost  of  the  project,  and  the 
expected  completion  date.  Based  on  this,  it  appears  that  the  results  of  this 
study  indicate  that  graphical  information  is  more  effective  in  aiding  program 
managers  evaluate  a  contractors  performance  when  accuracy  is  not 
important.  If  accuracy  is  important,  a  combination  of  graphical  and  tabular 
information  should  be  used. 

Decision  speed  was  the  next  area  examined.  The  results  of  the  statistical 
analysis  showed  that  there  was  no  significance  difference  in  speed  between 
graphical  and  tabular  reports.  Therefore,  in  terms  of  decision  speed  either 
method  of  information  presentation  may  be  used  and  the  results  will  be  the 
same. 

According  to  the  statistical  analysis,  confidence  and  difficulty  were  not 
related  to  the  mode  of  presentation  that  was  used.  On  the  contrary,  the 
findings  revealed  that  the  perceived  level  of  confidence  is  inversely  related 
to  years  of  Air  Force  experience.  In  addition,  the  pearson  correlation 
analysis  showed  that  as  the  scores  on  the  group  embedded  figures  test 
increased  the  subject  had  more  confidence  in  his  evaluation,  and  less 
difficulty  in  completing  the  task,  but  the  mode  of  presentation  had  nothing 
to  do  with  this  result. 

Individual  preference's  play  an  important  role  when  choosing  what  mode 
of  presentation  to  use.  Information  should  be  presented  to  a  manager  in  a 
the  form  that  he  will  get  the  most  benefit  from.  Based  on  the  analysis  of  the 
ratings  given  to  each  report,  graphs  were  found  more  useful,  and  tables 
more  accurate.  No  significant  difference  was  found  in  the  areas  of  difficulty 


and  report  satisfaction.  Siity-one  percent  of  the  individuals  test  •  ?.id  they 
would  prefer  to  have  the  information  in  graphical  form.  However,  this 
number  could  change  if  accuracy  was  viewed  as  the  most  important  factor 

Conclusion 

The  results  of  the  analysis  show  that  there  is  no  clear  cut  answer  to  the 
research  question.  The  study  showed  that  the  mode  of  presentation  had  no 
significant  effect  on  decision  speed,  confidence,  and  difficulty.  Decision 
quality  was  the  only  effectiveness  measure  that  was  affected  by  the  mode  of 
presentation.  In  terms  of  decision  quality,  the  results  indicate  that  the  task 
to  be  accomplished  had  an  effect  on  which  mode  was  most  effective.  If 
accuracy  is  desired,  then  tabular  information  would  have  to  be  chosen  as  the 
mode  of  presentation.  If  trend  detection  is  more  important,  then  graphs 
would  prove  to  be  more  effective.  Perhaps  a  better  alternative  would  be  to 
present  the  program  manager  with  both  modes  of  presentation.  This  will 
enable  tables  to  be  used  when  accuracy  is  needed,  and  graphs  when  trend 
detection  is  desired. 

In  conclusion,  it  can  not  be  determined  using  these  results  which  mode  of 
presentation  is  more  effective  in  aiding  a  program  manager  evaluate  a 
contractor  s  performance  when  C/SCSC  is  used.  The  mode  of  presentation 
needs  to  fit  the  task  to  be  accomplished,  and  there  are  several  tasks  required 
to  evaluate  a  contractors  performance.  Instead  of  choosing  one  mode  for  the 
entire  situation,  it  would  be  best  to  look  at  each  task  individually  and  choose 
the  best  mode  of  presentation  for  that  task.  Therefore,  the  best  solution 
would  be  to  use  both  modes  of  presentation  to  evaluate  a  contractor's 
performance. 


Recommendations 

This  research  only  examined  one  aspect  of  a  program  manager's  job. 
Since  this  research  showed  that  the  task  to  be  accomplished  was  an 
important  factor,  each  aspect  of  a  program  manager's  job  needs  to  be  looked 
at  separately  to  determine  which  mode  of  presentation,  or  combination  of 
modes,  is  best  suited  for  that  situation.  Other  possible  areas  of  study  are  in 
planning,  programming,  and  budgeting,  manpower  requirements,  scheduling, 
trade  off  analysis  (performance  vs  cost  vs  schedule),  and  cost  analysis. 
Additional  study  may  also  be  done  in  the  area  of  C/SCSC  to  determine  what 
combination  of  modes,  tables  and  graphs,  will  produce  the  best  results. 

Another  area  to  consider  is  the  effect  of  altering  the  graph's  features.. 
Perhaps  there  are  better  ways  to  graph  the  information  than  the  ones  used 
in  this  study.  For  example,  the  use  of  color  graphics  may  produce  better 
results.  Also,  the  scale  of  the  graphs  could  also  be  changed,  possibly 
resulting  in  better  accuracy  when  the  information  is  portrayed  graphically. 

Research  in  these  areas  are  needed  because  of  the  increased  availability 
of  computer  graphics  capabilities  for  office  workstations.  Proponents  of 
computer  graphics  are  saying  that  graphics  will  lead  to  better  decision 
making,  but  this  has  yet  to  be  proven.  The  cost  of  these  new  capabilities 
need  to  be  warranted  before  the  systems  are  purchased. 
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Appendix  A:  Test  Scenarios 


Test  One:  Cover  Sheet 

The  Contractor  has  supplied  you  with  the  following  information  pertaining  to 
project  X.  Elements  A.B,  and  C  make  up  the  project  X.  Answer  the  following 
questions  based  on  the  information  supplied.  You  will  have  20  minutes  to 
complete  the  test. 

Key  Terms 

BCWS  -  Budgeted  Cost  Work  Scheduled  -  the  amount  of  work  that  should  have 
been  completed  to  date  and  the  amount  it  should  have  cost. 

BCWP  -  budgeted  cost  work  performed  -  the  amount  of  work  that  has  been 
completed  to  date  and  the  amount  that  the  work  should  have  cost. 

ACWP  -  act  jal  cost  work  performed  -  the  actual  cost  of  the  work  that  has  been 
completed  to  date. 

BCWP-BCWS  -  schedule  variance 
BCWP- ACWP  -  cost  variance 

Negative  variances  implies  behind  schedule  or  over  cost. 


Test  One:  Tabular 


PERFORMANCE  DATA  FOR  MAY 
(CUMULATIVE  TO  DATE) 


ELEMENT 

BCVS 

BCWP 

ACVP 

SCHEDULE 

VARIANCE 

COST 

VARIANCE 

A 

1200 

1230 

1290 

50 

(40) 

B 

2400 

1840 

2000 

(560) 

(160) 

C 

2000 

1894 

1939 

(106) 

(45) 

TOTAL 

3600 

4984 

5229 

(616) 

(245) 

Using  the  information  above  answer  questions  1-9. 

1.  Is  the  contractor  behind  or  ahead  of  schedule? 

2.  Is  the  contractor  over  or  under  cost? 

3.  What  elements  are  over  cost? 

under  cost? 

4.  What  elements  are  behind  schedule? 

ahead  of  schedule? 

3.  What  is  the  total  actual  cost  of  the  work  performed? 

6.  What  is  the  total  budgeted  cost  for  the  work  performed? 

7.  What  is  the  total  budgeted  cost  for  the  work  scheduled? 

8.  What  is  the  total  schedule  variance  in  $? 

9.  What  is  the  total  cost  variance? 


%  COST/SCHEDULE  VARIANCE 
(Cumulative) 


%  SCHEDULE 

Dec 

jan 

Feb 

Mar 

April 

May 

VARIANCE 

-1.0 

-4.9 

-9.3 

-12.3 

-14.9 

-11.0 

%  COST 

VARIANCE 

-4.8 

-33 

-30 

-31 

-3A 

-4.9 

Use  the  above  information  to  answer  questions  10-12. 

10.  Is  there  any  relationship  between  the  %cost  and  %schedule  variance?  If 
yes  what  is  it.  (such  as,  as  one  goes  up,  the  other  goes  down) 

1 1.  What  is  the  trend  in  the  %schedule  variance  (getting  better,  worse,  staying 
the  same,  or  no  trend)? 

12.  What  is  the  trend  in  the  %cosi  variance  (getting  better,  worse,  staying  the 
same,  or  no  trend)? 
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MONTHLY  TOTALS 
(Cumulative) 


Dec 

Jan 

Feb 

Mar 

Apr 

May 

BCWS 

1200 

2230 

3040 

3973 

4730 

3600 

BCWP 

1188 

2119 

2731 

3483 

4040 

4964 

ACWP 

1243 

2232 

2889 

3662 

4273 

3229 

Use  the  above  information  to  answer  questions  13-M 

13.  Is  there  a  month(s)  that  the  contractor  was  under  cost  (ACWP  <  BCWP)? 

If  yes,  which  month? 

14.  Is  there  a  month(s)  where  the  contractor  was  ahead  of  schedule  (BCWP 
BCWS)? 

If  yes.  which  month? 


After  completing  questions  1-14,  raise  your  hand  so  the  time  it  took  you  to 
complete  the  test  can  be  recorded. 


Test  One:  Graphical 


Cast  VriMci 

Schadala  Variamca 
Actual  Cast  W ark  parfarata4 


Cast  Vark  PirfaraH 
Cast  Wark  Scfca4ala4 

-i 


Dallars  taThaasaads 


Using  the  information  above  answer  questions  1-7. 

1.  Is  the  contractor  behind  or  ahead  of  schedule? 

2.  Is  the  contractor  over  or  under  cost? 

3.  What  is  the  actual  cost  of  the  work  performed? 

4.  What  is  the  budgeted  cost  for  the  work  performed? 
3.  What  is  the  budgeted  cost  for  the  work  scheduled? 

6.  What  is  the  schedule  variance  in  $? 

7.  What  is  the  cost  variance? 
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Using  the  charts  on  the  following  page,  answer  questions  8-9. 


8.  What  elements  are  over  cost? 

under  cost? 

9.  What  elements  are  behind  schedule? 

ahead  of  schedule? 


Parfarmaca  Rapart 
Elamaal  A 


Dali 


CmL  Variance 


Psrfsraiaacs  Rapart 
ElamaaLB 

Hay.  CasMlattva  ta 
Bala 


ScfeaBala  Variaaca 


Aclsal  Cast  Wart  ParfanaaB 


BaBfataB  Cast  Wart  Parfar 


BaBfalaB  Cast  Wart  SchaBaW 


ScfeaBala  Variaaca 


Actaal  Cast  Wart  ParfarawB 


Dalian  la 


Cast  Variaaca 


Parfarataaca  Rapart 

Elamaat  C 

flay.  Caawlattva  ta 

Bats 


BaBfalaB  Cast  Wart  ParfaraiaB  j 


BaBfalaB  Cast  Wart  ScksBalsB 


Dalian  ia  UtaasasBs 


S  SclMdula/CMt  Variants. 
Cumulative 


™  Schedule  Variance 
•"  Cost  Variance 


Use  the  above  information  to  answer  questions  10-12. 


10.  Is  there  any  relationship  between  the  %cost  and  %schedule  variance?  If 
yes.  what  is  it.  (such  as.  as  one  goes  up,  the  other  goes  down) 


1 1.  Is  there  any  trend  in  the  schedule  variance?  If  yes,  what  is  it?  (getting 
better,  staying  the  same,  getting  worse,  or  no  trend) 


12.  Is  there  any  trend  in  the  cost  variance?  If  yes,  what  is  it?  (getting  better, 
staying  the  same,  getting  worse,  or  no  trend) 


ninthly  Tatals.  CaMalativa 


Thousands 


■  BadyaLad  cast  Work 
ScImAiM 

□  Badyotod  Cast  Work 
Par  far  mad 

B  Actaal  Cast  Work 
Par  far  Mad 


Months 


Use  the  above  information  to  answer  questions  13-14 

13.  Are  there  any  month(s)  that  the  contractor  was  under  cost  (ACWP< 
BCWP)? 

If  yes,  which  month? 


14.  Are  there  any  month(s)  where  the  contractor  was  ahead  of  schedule 
(BCWP  >  BCWS)? 

if  yes,  which  month? 

After  completing  questions  1-14,  raise  your  hand,  so  the  time  it  took  you  to 
complete  the  test  can  be  recorded. 
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Test  Two:  Cover  Sheet 

The  Contractor  has  supplied  you  with  the  following  information  pertaining  to 
project  X.  Elements  A3,  and  C  make  up  the  project  Y.  Answer  the  following 
questions  based  on  the  information  supplied.  You  will  have  20  minutes  to 
complete  the  test. 

key  Terms 

BCWS  -  Budgeted  Cost  Work  Scheduled  -  the  amount  of  work  that  should  have 
been  completed  to  date  and  the  amount  it  should  have  cost. 

BCWP  -  budgeted  cost  work  performed  -  the  amount  of  work  that  has  been 
completed  to  date  and  the  amount  that  the  work  should  have  cost. 

ACWP  -  actual  cost  work  performed  -  the  actual  cost  of  the  work  that  has  been 
completed  to  date. 

BCWP-BCWS  -  schedule  variance 
BCWP- ACWP  -  cost  variance 

Negative  variances  implies  behind  schedule  or  over  cost. 


Test  Two.  Tabular 

PERFORMANCE  DATA  FOR  April 
(CUMULATIVE  TO  DATE) 


ELEMENT 

BCYS 

BCWP 

ACWP 

SCHEDULE 

VARIANCE 

COST 

VARIANCE 

A 

2300 

2197 

2200 

(303) 

(3) 

B 

1130 

1010 

1400 

(140) 

(390) 

C 

1700 

1493 

1688 

(207) 

(195) 

TOTAL 

3330 

4700 

3288 

(630) 

(388) 

Using  the  information  above  answer  questions  1-9. 

1.  Is  the  contractor  behind  or  ahead  of  schedule? 

2.  Is  the  contractor  over  or  under  cost? 

3.  What  elements  are  over  cost? 

under  cost? 

4.  What  elements  are  behind  schedule? 

ahead  of  schedule? 

5.  What  is  the  total  actual  cost  of  the  work  performed? 

6.  What  is  the  total  budgeted  cost  for  the  work  performed? 

7.  What  is  the  total  budgeted  cost  for  the  work  scheduled? 

8.  What  is  the  total  schedule  variance  in  $? 


9.  What  is  the  total  cost  variance? 


%  COST/SCHEDULE  VARIANCE 
(Cumulative) 


%  SCHEDULE 

Dec 

Jan 

Feb 

Mar 

Apr 

VARIANCE 

-350 

-24.5 

-258 

-14.0 

-12.1 

%  COST 

VARIANCE 

-5.0 

-55 

2.0 

-12.0 

-12.5 

Use  the  above  information  to  answer  questions  10-12. 

10.  Is  there  any  relationship  between  the  Xcost  and  X schedule  variance?  If 
yes,  what  is  it.  (such  as,  as  one  goes  up,  the  other  goes  down) 

11.  Is  there  any  trend  in  the  schedule  variance?  If  yes,  what  is  it?  (getting 
better,  staying  the  same,  getting  worse,  or  no  trend) 

12.  Is  there  any  trend  in  the  cost  variance?  If  yes,  what  is  it?  (getting  better, 
staying  the  same,  getting  worse,  or  no  trend) 


MONTHLY  TOTALS 
(cumulative) 


Dec  Jan  Feb  Mar  Apr 

BCWS  1500  2450  3300  4475  5350 

BCWP  970  1850  2450  3850  4700 

ACWP  1018  1952  2400  4312  5288 

Use  the  above  information  to  answer  questions  1 3-H 


13.  Are  there  any  month(s)  that  the  contractor  was  under  cost  (ACWP< 
BCWP)? 

If  yes,  which  month? 

14.  Are  there  any  month(s)  where  the  contractor  was  ahead  of  schedule 
(BCWP  >  BCWS)? 

if  yes,  which  month? 

After  completing  questions  1-14,  raise  your  hand,  so  the  time  it  took  you  to 
complete  the  test  can  be  recorded. 


Test  Two:  Graphical 


Parformanco  Measuramont  Raport  for  April. 
Cwnalativo  la  dalo 
Tatal  A.B.C 


Cost  Varianca 
Schodulo  Variance 
Ac  tool  Cost  Work  porforinod 
Badfotod  Cost  Work  Par  for  mad 
Bodfotod  Cost  Work  Schodolod 


m 

■ran 

mci- 

lllp 

-( 


Dollars  la  thousands 


Using  the  information  above  answer  questions  1-7. 

1.  Is  the  contractor  behind  or  ahead  of  schedule? 

2.  Is  the  contractor  over  or  under  cost? 

3.  What  is  the  actual  cost  of  the  work  performed? 

4.  What  is  the  budgeted  cost  for  the  work  performed? 

5.  What  is  the  budgeted  cost  for  the  work  scheduled? 

6.  What  is  the  schedule  variance  in  S? 


7.  What  is  the  cost  variance? 


Using  the  charts  on  the  following  page,  answer  questions  8-9 


8.  What  elements  are  over  cost? 

under  cost? 

9.  What  elements  are  behind  schedule? 

ahead  of  schedule? 


Cast  Varimca 


PtrfirMKi  Report 
Element  A 

April.  Cumulative  la  Data 


Schedule  Variance 


Actnal  Cast  Work  Performed 


BWf«M  Cast  Work  Per  farmed 


Badfotod  Cast  Work  Scheduled 

i - 

-i 


s 


Dollars  In  thousands 


Dollars  ia  thoasaatfs 


Use  the  above  information  to  answer  questions  10-12. 


10.  Is  there  any  relationship  between  the  %cost  and  %schedule  variance?  If 
yes,  what  is  it.  (such  as,  as  one  goes  up,  the  other  goes  down) 


11.  Is  there  any  trend  in  the  schedule  variance?  If  yes,  what  is  it?  (getting 
better,  staying  the  same,  getting  worse,  or  no  trend) 


12.  Is  there  any  trend  in  the  cost  variance?  If  yes,  what  is  it?  (getting  better, 
staying  the  same,  getting  worse,  or  no  trend) 


1 

'i t 


I 


8 


Months 


Use  the  above  information  to  answer  questions  13-14 

13-  Are  there  any  month(s)  that  the  contractor  was  under  cost  (ACWP< 
BCWP)? 

If  yes,  which  month? 

14.  Are  there  any  month(s)  where  the  contractor  was  ahead  of  schedule 
(BCWP  >  BCWS)? 

it  yes,  which  month? 

After  completing  questions  1-14,  raise  your  hand,  so  the  time  it  took  you  to 
complete  the  test  can  be  recorded. 


Appendix  B:  Rating  Scales  and  1 


lestionnaire 


Confidence  and  Difficulty  Rating  Scales 


Use  the  following  scale  and  rate  the  amount  of  confidence  you  have  in  your 


answers  to  the  above  questions. 


No 

Confidence 


Moderate 

Confidence 


Extreme 

Confidence 


4  5 


Use  the  following  scale  to  rate  the  level  of  difficulty  you  had  in  answering  the 
questions. 


Not 

Difficult 


Moderately 

Difficult 


Extremely 

Difficult 


vsxb&bssxbsrrr 


Report  Rating  Scales  and  Demographics  Questionnaire 
Rate  each  report  using  the  following  scales 

1.  How  useful  was  the  report  format  in  helping  you  answer  the  question? 


Graphical 

Very  Not 

useful  _ useful 


12  3  4  5 

Tabular 

Very 

userul 

6 

7 

Not 

useful 

1  2  3  4  5 

2.  How  accurate  was  the  report  format? 

6 

7 

Graphical 

Very 

Not 

Accurate 

_ Accurate 

1  2  3  4  5 

6 

7 

Tabular 

Very 

Not 

Accurate 

Accurate 

1  2  3  4  5 

3-  How  dificult  was  the  report  to  understand? 

6 

7 

Graphical 

Not 

Very 

Difficult 

Difficult 

1  2  3  4  5 

6 

7 

Tabular 

Not 

Very 

Difficult 

Difficult 

1  2  3  4  5 

6 

7 
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'.VI'. 


4.  How  satisfied  were  you  with  the  report  format? 

Graphical 

Very  Dis¬ 
satisfied 

12  3  4  5  6  7 

Tabular 

Very 
Satisfied 

1  2  3  4  5  6  7 


Very  Dis¬ 
satisfied 


Very 

Satisfied 


5.  Which  format  did  you  like  better?  G  T 

6.  Which  one  would  you  prefer  if  you  had  to  do  the  test  again?  G  T 
BACKGROUND  INFORMATION 

What  is  your  current  AFSC  (or  civilian  equivalent)? 

How  many  years  of  experince  do  you  have  in  that  AFSC? 

How  long  have  you  been  in  the  AF,  or  worked  for  the  AF? 

Have  you  used  C/SCSC  before? 

If  yes  how  long? 

Have  you  ever  used  graphics  as  a  mode  of  information  presentation  before? 

Grade/Rank _ 

Sex _ 

Age _ 
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